Simplifying the Development and Deployment of MapReduce Algorithms

MapReduce algorithms can be difficult to write and test due to the accidental complexities involved with existing MapReduce implementations. Furthermore, the configuration details involved in running MapReduce algorithms within a cloud present a set of new challenges. Our research reveals that many details of cloud configuration can be hidden from programmers in an automated and transparent manner. Using concepts from software engineering, we have increased the ease of use for implementing MapReduce algorithms by creating a lightweight domain-specific language (DSL). Additionally, we created a plug-in for the Eclipse integrated development environment (IDE) based on this DSL to automate and hide many cloud configuration details. The goal of the combination of our IDE and DSL is to improve the efficiency and effectiveness for programmers to develop MapReduce algorithms for cloud computing.